CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures

Learning commonsense casual and temporal relation between events is one of the major steps towards deeper language understanding. This is crucial for understanding stories and narrative structure learning. In this paper we introduce a novel semantic annotation framework, called Causal and Temporal Relation Scheme (CaTeRS), which is a prerequisite for learning narrative structures of stories. This scheme is unique in capturing both temporal and causal aspects of inter-event relations. By annotating a total of 1,600 sentences in 320 five-sentence short stories sampled from ROCStories corpus, we demonstrate that these stories are indeed full of causal and temporal relations. Furthermore, we show that the CaTeRS annotation scheme enables high inter-annotator agreement for broad-coverage event entity annotation and moderate agreement on semantic link annotation.